A sparse Markov chain approximation of LQ-type stochastic control problems

We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind admit a dual formulation in terms of linear boundary value problems. We show that the discretized linear problem is dual to a Markov decision problem, prove an $L^{2}$ error bound for the general scheme and discuss the sparse discretization using a basis of so-called committor functions as a special case; the latter is particularly suited when the dynamics are metastable, e.g., when controlling biomolecular systems. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation

Diffusion maps for Lagrangian trajectory data unravel coherent sets

Dynamical systems often exhibit the emergence of long-lived coherent sets, which are regions in state space that keep their geometric integrity to a high extent and thus play an important role in transport. In this article, we provide a method for extracting coherent sets from possibly sparse Lagrangian trajectory data. Our method can be seen as an extension of diffusion maps to trajectory space, and it allows us to construct “dynamical coordinates,” which reveal the intrinsic low-dimensional organization of the data with respect to transport. The only a priori knowledge about the dynamics that we require is a locally valid notion of distance, which renders our method highly suitable for automated data analysis. We show convergence of our method to the analytic transfer operator framework of coherence in the infinite data limit and illustrate its potential on several two- and three-dimensional examples as well as real world data. One aspect of the coexistence of regular structures and chaos in many dynamical systems is the emergence of coherent sets: If we place a large number of passive tracers in a coherent set at some initial time, then macroscopically they perform a collective motion and stay close together for a long period of time, while their surrounding can mix chaotically. Natural examples are moving vortices in atmospheric or oceanographic flows. In this article, we propose a method for extracting coherent sets from possibly sparse Lagrangian trajectory data. This is done by constructing a random walk on the data points that captures both the inherent time-ordering of the data and the idea of closeness in space, which is at the heart of coherence. In the rich data limit, we can show equivalence to the well-established functional-analytic framework of coherent sets. One output of our method are “dynamical coordinates,” which reveal the intrinsic low- dimensional transport-based organization of the data

Meshless discretization of LQ-type stochastic control problems

Abstract. We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind can be transformed into linear boundary value problems by a logarithmic transformation. We show that the discretized linear problem is dual to a Markov decision problem, the precise form of which depends on the chosen Galerkin basis. We prove a strong error bound in L2 for the general scheme and discuss two special cases: a variant of the known Markov chain approximation obtained from a basis of characteristic functions of a box discretization, and a sparse approximation that uses the basis of committor functions of metastable sets of the dynamics; the latter is particularly suited for high-dimensional systems, e.g., control problems in molecular dynamics. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation. 1. Introduction. A large body of research is concerned with the question: How well can a continuous diffusion in an energy landscape be approximated by a Markov jump process (MJP) in the regime of low temperatures? Qualitatively, the approxima

Transition manifolds of complex metastable systems: Theory and data-driven computation of effective dynamics

We consider complex dynamical systems showing metastable behavior but no local separation of fast and slow time scales. The article raises the question of whether such systems exhibit a low-dimensional manifold supporting its effective dynamics. For answering this question, we aim at finding nonlinear coordinates, called reaction coordinates, such that the projection of the dynamics onto these coordinates preserves the dominant time scales of the dynamics. We show that, based on a specific reducibility property, the existence of good low-dimensional reaction coordinates preserving the dominant time scales is guaranteed. Based on this theoretical framework, we develop and test a novel numerical approach for computing good reaction coordinates. The proposed algorithmic approach is fully local and thus not prone to the curse of dimension with respect to the state space of the dynamics. Hence, it is a promising method for data-based model reduction of complex dynamical systems such as molecular dynamics

Network Measures of Mixing

Transport and mixing processes in fluid flows can be studied directly from Lagrangian trajectory data, such as obtained from particle tracking experiments. Recent work in this context highlights the application of graph-based approaches, where trajectories serve as nodes and some similarity or distance measure between them is employed to build a (possibly weighted) network, which is then analyzed using spectral methods. Here, we consider the simplest case of an unweighted, undirected network and analytically relate local network measures such as node degree or clustering coefficient to flow structures. In particular, we use these local measures to divide the family of trajectories into groups of similar dynamical behavior via manifold learning methods

The Unruh-deWitt Detector and the Vacuum in the General Boundary formalism

We discuss how to formulate a condition for choosing the vacuum state of a quantum scalar field on a timelike hyperplane in the general boundary formulation (GBF) using the coupling to an Unruh-DeWitt detector. We explicitly study the response of an Unruh-DeWitt detector for evanescent modes which occur naturally in quantum field theory in the presence of the equivalent of a dielectric boundary. We find that the physically correct vacuum state has to depend on the physical situation outside of the boundaries of the spacetime region considered. Thus it cannot be determined by general principles pertaining only to a subset of spacetime.Comment: Version as published in CQ

MIST: A Simple and Efficient Molecular Dynamics Abstraction Library for Integrator Development

We present MIST, the Molecular Integration Simulation Toolkit, a lightweight and efficient software library written in C++ which provides an abstract in- terface to common molecular dynamics codes, enabling rapid and portable development of new integration schemes for molecular dynamics. The initial release provides plug-in interfaces to NAMD-Lite, GROMACS and Amber, and includes several standard integration schemes, a constraint solver, tem- perature control using Langevin Dynamics, and two tempering schemes. We describe the architecture and functionality of the library and the C and For- tran APIs which can be used to interface additional MD codes to MIST. We show, for a range of test systems, that MIST introduces negligible overheads for serial, shared-memory parallel, and GPU-accelerated cases, except for Amber where the native integrators run directly on the GPU itself. As a demonstration of the capabilities of MIST, we describe a simulated tempering simulation used to study the free energy landscape of Alanine-12 in both vacuum and detailed solvent conditions.Comment: Pre-print accepted for Computer Physics Communication